Regression analysis

A way to think and generalize the statistical models is breaking them into a stochastic and a systematic component.

$$\begin{aligned} Y_i \sim f(\theta_i, \alpha) \quad &\text{(stochastic)} \\ \theta_i = g(x_i, \beta)\quad &\text{(systematic)} \end{aligned}$$

where there is always an estimation uncertainty – the lack of knowledge of $\beta$ and $\alpha$ – and fundamental uncertainty (stochastic component).

• Linear regression
• Logistic regression
• Multiple regression
• Bayesian linear regression
• Panel analysis
• Zero inflated model

Endogeneity

Interaction

• A Tutorial on Interaction by Tyler J and Mirjam J
• Interactions by Matt Golder
• How Much Should We Trust Estimates from Multiplicative Interaction Models? Simple Tools to Improve Empirical Practice
• Understanding Interaction Models: Improving Empirical Analysis

Marginal effect

Marginal effect refers to “the slope of the regression surface with respect to a given covariate” (Leeper, 2017). In other words, it is about how much the dependent variable changes when we change the given independent variable. Of course this is not necessarily a constant. The slope can change based on the value of the independent variables. Thus we consider a representative marginal effect. A common method is calculating the “average marginal effects (AMEs)”, which is the mean of the marginal effect for every data point. Alternatively, the marginal effect can be calculated at the means of independent variables, “marginal effects at means (MEMs)”, or at a representative values (MERs). AMEs are most ‘data-driven’ and reflect the full data distribution.

• Leeper, 2017: Interpreting Regression Results using Average Marginal Effects with R’s margins